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1!j  ABSTRACT  (Mjxirriijni  200  wcirdi) 

We  have  been  working  at  the  NCCOSC  liD'I’&E  Division  (NRaD)  to  develop  new  methods  of  assisting  Navy  decision  makers 
in  real  time  command  and  control  decision-making  situations.  Our  work  has  focused  directly  on  the  incorfioration  of  new  kinds 
of  knowledge  based  computer  tools  into  shipboard  decision  support  systems.  As  part  of  that  work,  we  have  had  the  opportunity 
to  examine  various  approaches  to  influencing  tinman  decision  makers  through  interface  manipulations  The  general  conclusion 
we  have  reached  is  that  interfaces  used  by  expert  decision  makers  m  real  time  situations  should  make  only  limited  use  of  tech¬ 
niques  such  as  direct  manipulation  that  require  users  to  interact  v/ith  the  system.  It  follows  that  efforts  to  provide  a  critiquing 
mechanism  should  emphasize  critics  that  operate  automatically  without  waiting  for  the  user  to  request  assistance  However,  we 
are  also  looking  for  an  interface  that  does  not  automatically  change  displays  in  ways  that  users  may  finrl  intrusive,  or  otherwi.se 
objectionable.  To  address  both  having  automatic  criticism  and  non-intrusive  display  changes,  we  are  looking  for  ways  to  provide 
critiquing  that  is  transparent  to  the  user. 

This  paper  de.scribes  ideas  about  appropriate  ways  to  make  use  of  ideas  from  work  involving  critics 
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IntixKluction. 

We  have  been  working  at  the  NCCOSC  RDT&E  Division  (NRaD)  to  develop 
new  methods  of  assisting  Navy  decision  makers  in  real  time  command  and 
control  decision-making  situations.  Our  work  has  focused  directly  on  the 
incorporation  of  new  kinds  of  knowledge  based  computer  tools  into 
shipboard  decision  support  systems.  As  part  of  that  work,  we  have  had  the 
opportunity  to  examine  various  approaches  to  influencing  human  decision 
makers  through  ijiterface  manipulations.  The  general  conclusion  we  have 
reached  is  that  interfaces  used  by  expert  decision  makers  in  real  time 
situations  should  make  only  limited  use  of  techniques  such  as  direct 
manipulation  that  require  users  to  interact  with  the  system.  It  follows  that 
efforts  to  provide  a  critiquing  mechanism  should  emphasize  critics  that 
operate  automatically  without  waiting  for  the  user  to  request  assistance. 
However,  we  are  also  looking  for  an  interface  that  does  not  automatically 
change  displays  in  ways  that  users  may  find  intrusive,  or  otherwise 
objectionable.  To  address  boi,n  having  automatic  criticism  and 
nonintrusive  display  changes,  we  are  trying  to  find  ways  to  provide  some 
critiquing  in  a  way  that  is  transparent  to  the  user. 

Underlying  Problems. 

The  research  at  NRaD  is  focused  on  the  testing  of  new  ideas  about  the  use  of 
knowledge  based  decision  aids  in  tactical  situations  where  decisions  must 
be  made  under  severe  time  constraints,  based  on  incomplete  or  ambiguous 
information.  This  work  is  funded  by  the  Office  of  Naval  Technology  under  a 
project  called  TActical  Decision  Making  Under  Stress  (TADMUS). 

The  primary  goals  of  the  research  are  to  seek  solutions  to  the  following 
problems.  First,  there  is  the  problem  of  trying  to  find  ways  to  remedy 
human  errors.  Research  indicates  that  human  decision  makers 
sometimes  make  decision  errors  as  a  result  of  one  or  more  cognitive  biases 
(Kahneman  &  Tversky,  1982).  However,  there  has  been  little  done  to  test 
approaches  to  providing  remedies  for  such  errors.  Our  effort  is  aimed  at 
trying  to  provide  a  remedy  through  the  use  of  knowledge  based  computer 
tools.  Other  work  in  the  TADMUS  project  is  looking  into  new  ideas  in 
training  as  a  remedy. 

A  second  problem  is  to  develop  tools  that  can  be  accepted  by  the  users.  This 
problem  is  more  serious  in  our  situation  than  in  other  areas  involving  the 


use  ol' k!i()wl(>(l^(('  based  ( (‘clmolof^y,  bt'caiise  in  out'  case  decisions  will  oIUmi 
hav('  life  and  deal.li  cons('(]ii('nc(‘s.  It  is  essential,  tlu'n,  that,  ust'cs  I'nlly 
iindetstand  vvlial,  t.he  d('cisi()n  support,  syst(!m  can  do  for  them,  as  w('ll  as 
what,  tlu'  syst('m  cannot  do.  ddu^se  an'  situations  in  which  it,  is  (h'dnilcly  not, 
h('liev('d  to  h(>  d(>sirahl('  (or  us(U's  to  have  the  impression  (liat  tin*  computer 
can  make  th('  final  decisions  on  its  own. 

A  third  prolikmi  lies  in  determining  iiow  best  to  jiresent  information  to  tin* 
us(!rs.  It  is  not  enough  to  produc<!  tools  that  can  prswide  critical  kinds  of 
information  to  users.  That  information  must  h(?  structured  in  ways  I  hat, 
make  it  readily  accessible.  Moreover,  in  a  real  time  setting  it  is  important 
to  minimize  the  degree  to  wliieh  the  loo!  itself  may  become  intrusive  in 
ways  that  can  degrade  performance. 

Deci.sion-making  strategies. 

Initial  research  for  TADMUS  was  aimed  at  identifying  decision-making 
strategies  used  by  expert  Navy  decision  makers.  That  research  has  pointed 
to  the  use  of  two  primary  naturalistic  decision-making  strategies: 
recognition  primed  decision  making  (RPD)  (Klein,  1989)  and  explanation 
based  decision  making  (Pennington  &  Hastie,  1988). 

Based  on  the  finding  that  those  two  strategies  are  predominant,  two 
decision  support  tools  have  been  prototyped  that  try  to  mode!  the  strategies. 
One  tool  uses  a  template  based  approach  to  model  the  RPD  idea  that  most 
expert  decision  making  is  based  on  matching  the  features  of  the  current 
situation  to  features  of  prior,  stored,  situation  templates.  The  other  tool  is 
called  SABER  (Situation  Assessment  By  Explanation  based  Reasoning), 
and  it  models  the  explanation  based  decision  making  strategy  in  which 
explanations  are  constructed  to  account  for  available  information,  and 
decisions  are  based  on  a  determinationmf  the  degree  of  plausibility  of  the 
explanations  (Hair  et  al,  1992). 

The  hypothesis  behind  constructing  tools  that  model  the  user’s  own 
decision  making  strategies  is  that  the  use  of  those  strategies  should  enable 
the  tools  to  provide  the  kind  of  information  that  will  be  most  useful  to  the 
user,  in  a  way  that  will  be  readily  comprehensible.  Tied  into  that  idea  is  the 
belief  that  the  decision  support  system  should  be  able  to  suggest  to  the  user 
where  errors  could  arise,  or  could  guide  the  user  away  from  making 
errors.  This  latter  kind  of  information  is  in  the  nature  of  offering 
suggestions,  but  is  kept  indirect  both  so  it  will  not  be  intrusive  in  the  real 
time  situati  on  and  so  it  will  not  be  seen  as  a  case  of  the  computer  trying  to 
make  the  decisions.  Part  of  our  hypothesis  is  that  some  cognitive  biases 
may  be  remediable  through  such  devices  as  always  presenting  the  user 
with  alternative  explanations,  and  highlighting  for  the  user  where  the 
weaknesses  are  in  the  explanations. 

Integrating  the  tools  into  a  decision  support  system. 

Having  developed  tools  that  model  key  decision  making  strategies,  the 
problem  remains  of  developing  an  interface  to  exploit  the  use  of  those 


modi'l.s.  Ill  llu'  si'l-lii)|.j  oC  Navy  command  and  I’onlrol  operations,  (ho  roal 
litiK'  nat.uri'  of' t.lu'  prohlom  ends  up  drivin^^  a  nnmix'r  ordc'cisions 
rc^^ardinp  (ho  intm't'.vci'. 

'I'ho  om[)hasis  on  rmd  time  oporations  has  lo(i  to  a  pt-iu'i'al  idimination  of 
Coatufos  that  roquiro  inti'iaction  with  the  user.  'I'hat  elimination  leads  to  a 
severe  simplification  of  tool  capahilities  from  th(‘  orif^inal  prototypt's.  Those 
{irototypes  had  made  considerahle  use  of  a  variety  of  direct  manipulatiiin 
techniques  that  appear  impractical  Tor  the  final  real  time  system  hecause 
they  cause  a  loss  of  time  in  situations  where  time  is  at  a  premium.  A  direct 
conclusion  resulting  from  the  real  time  consideration  is  that  the  interface 
needs  to  provide  information  concisely,  and  preferably  automatically. 
However,  pulling  against  that  approach  is  the  need  to  he  able  to  let  users 
have  some  ability  to  select  different  information  displays  according  to  their 
own  belief  as  to  what  is  most  useful  for  them  to  see  at  any  given  time.  In 
addition,  the  limited  availability  of  screen  space  leads  to  the  need  to  provide 
for  some  user  driven  mechanisms  for  changing  the  displays. 

Use  of  critics  in  this  setting. 

One  general  approach  to  enhancing  user  performance  is  to  provide  the  user 
with  some  form  of  criticism  of  the  user’s  ongoing  decision  making 
processes.  A  critic  facilit}'  could  attempt  to  focus  the  user’s  attention  on 
data  that  might  be  overlooked  due  to  cognitive  biases.  It  could  also 
highlight  alternative  analyses  as  a  way  of  keeping  the  user  aw'are  of  the  fact 
that  in  ambiguous  situations  different  analyses  should  be  considered. 

One  suggested  approach  has  been  to  incorporate  an  extensive  model  of  the 
user  v/ithin  the  system.  The  idea  is  that  such  a  model  can  enable  the 
system  to  form  a  judgement  based  on  current  user  actions  as  to  what  the 
user’s  cognitive  focus  is.  The  system  can  then  determine  what  kinds  of 
information  can  most  usefully  be  displayed  at  that  time.  This  kind  of 
system  would  act  as  a  critic  in  the  sense  that  it  would  be  making 
judgements  as  to  what  the  user  ought  to  be  doing,  and  would  provide 
information  intended  to  guide  the  user  to  doing  such. 

We  believe  that  approach  merits  further  study,  but  are  not  pursuing  it  in 
the  short  term  for  a  number  of  reasons.  First,  building  and  testing  the  user 
models  is  not  a  short  term  effort  in  it.self.  Second,  it  is  not  cicar  that  this 
approach  can  be  successfully  incorporated  into  a  real  time  system.  Third, 
there  is  a  current  of  thought  in  the  TADMUS  project  that  the  system  should 
not  actively  initiate  display  changes.  As  a  result,  the  short  term  eftbrt  is  to 
explore  approaches  in  which  any  changes  relating  to  completely  changing 
a  set  of  data,  as  opposed  to  merely  updating  the  current  data,  must  be  user 
initiated. 

The  problem  then  becomes  one  of  how  to  display  information  in  more  static 
displays  that  can  be  expected  to  po.sitiveIy  influence  user  decision  making. 
Part  of  our  approach  is  to  structure  the  displays  such  that  suggestions  are 
always  shown  about  alternative  conclusions.  Also,  where  the  RPD  tool 


fitids  (cinplalc  malclu's  of  siiriu'icMil.  inl.c'i'csl  lo  hi'iii);  to  t  he  users’  al  l  (Mil  ion, 
tile  ((‘in|)lal,('  i tifbrmal.ioii  is  shown  in  a  praphienl  way  that,  not  only  pivi's 
tiie  l.ool’s  own  ('st  imati'  oC  how  elosely  t  he  i.emplalc  inatclH's  Hu-  current 
sil.uation,  hut  also  ('iieourapes  users  (,o  form  theii'  own  ('slimali's.  Anolluu' 
k(‘y  part  of' the  desifpi  f'oeusi's  on  always  inainlaininp  displays  of  hist-oi'iiad 
information  as  a  way  oftr-yiii^  to  piawi'iit  iisiu's  fi'oin  losing  track  of  ridcwant 
piec('s  of  data. 

We  helievi*  that  througli  th(‘se  kinds  of  displays  the  lU'gative  (d'h'cts  of 
cognitive  biases  will  not  he  completely  eliminated,  luit  that  such  effects  can 
he  reduced.  Thus,  the  fact  that  alternative  explanations  are  always 
suggested  is  expected  to  act  as  a  critic  that  continually  points  up  possible 
weaknesses  in  each  possible  explanation.  The  continual  display  is  also 
intended  to  reduce  the  likelihood  that  the  user  will  focus  too  (|uickly  on  one 
explanation  without  giving  due  consideration  to  the  others.  We  believe  that 
these  kind  of  techniques  can  effectually  act  as  a  critiquing  mechanism  to 
enable  users  to  make  decisions  more  accurately  and  quickly. 

Conclusion. 

Our  work  on  developing  new  approaches  to  real  time  decision  support 
systems  has  led  to  some  mw  ideas  about  appropriate  ways  to  make  use  of 
ideas  from  work  involving  critics.  Basically,  we  conclude  that  what  is 
needed  is  an  interface  designed  to  present  data  in  ways  that  suggests 
alternative  interpretations,  and  also  highlights  possible  weaknesses  in  the 
interpretations  In  this  way,  a  kind  of  criticism  is  offered  to  the  user,  but 
the  criticism  is  not  direct  and  is  not  intrusive. 
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